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FOREWORD 



Within the last decade the concept of child advocacy has been set forth to 
insure that needed services, programs and resources be available for every 
child. As part of this bold and innovative idea the Joint Commission on Mental 
Health of Children Report (1970) made as its major recommendation the 
development of a nationwide system of child advocacy. 

I n North Carolina, as in every other state, we have sought to deal with 
the realities and implications of this commitment. The Child Advocacy System 
Project , one of six in the nation funded by the Joint Committee on Child 
Advocacy of the Bureau of Education for the Handicapped and the National 
Institute of Mental Health, was administered by the Learning Institute of 
North Carolina. The Learning tnstitute, with its mandate to seek new 
solutions to North Carolina's most pressing education problems, has sought 
through this project to demonstrate effective and systematic means through 
which community resources can be united on behalf of children and youth. 

A Matter of Service is one result of the Child Advocacy System 
Project. It is hoped that this publication will assist parents, citizen and 
professional groups, and other interested parties as they seek to facilitate 
each child's development. 

Another handbook, To Protect and Respect, was also produced as a 
result of our experienc .s in the Child Advocacy System Project. It describes 
methods which a group of community citizens use to find out how well 
services are provided to children. The methods described in each handbook 
serve as different ways to assure that a community is fulfill ing its responsibility 
to its youngest citizens. 



John R. B,; Hawes, Jr. 
Executive Director 



PREFACE 



Thismanual was produced as a result of the Child Advocacy System Project 
(C ASP). The general purpose of C ASP was to learn more about the concept 
of child advocacy, and about its practical application in a neighborhood 
where children live. It was clear from the outset of the project that one 
important function of child advocacy is to pay careful attention to the way in 
which services for children are provided. The objective of this function is to 
look carefully at the quality and adequacy of child service. If some part of the 
service network is found to be insufficient , then efforts can be made to correct 
problems or improve services. 

There was a strong belief among project staff and the people we worked 
with in the neighborhood and community that the function of looking at the 
child service network was partly the responsibility of the citizens of the 
community. This belief came from parents, other citizens concerned about 
children AND from professionals, themselves serving children. 

Thismanual was produced to help citizens carry out this function. It 
serves as a reference and guide to the process of lookingat child-serving 
agencies. The process is called monitoring. Monitoring is carried out by 
systematically collectingandinterpretingcertain kinds of information about 
agency operations. 

This manual can be used by a variety of groups having an interest in 
children. It can be used by civic clubs, fraternal organizations, or groups of 
community volunteers, who want to carry out agency monitoring as an 
important community program. It can be used by citizen groups working 
closely with child serving agencies, who by working together can support 
joint efforts to improve child services. It can also be used as a guide for 
agencies, themselves, in efforts to look closely at how well they are providing 
service they are mandated to provide. 

While an individual reader may find the material helpful, the monitoring 
process described herein requires involvement of a group of people. This guide 
does not require group members to be professionally trained, but professional 
help is recommended to select indicators, design data collection instruments, 
and analyze and interpret data. 
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WHAT^S IT ALL ABOUT? 

Introduction To Child Advocacy And Agency 
Monitoring 



Child advocacy is , first , a point of view which emphasizes the rights of children 
to having their basic health , housin^^, nittritionaK and clothing needs met as 
well as an opportunity to develop socially , emotionally, and educationally to 
their highest potential. Therefore, the purpose of child advocacy is to 
advocate for children in ensuringand protecting these rights and reminding 
communities of their responsibilities to their own children. 

Secondly , child advocacy is m active process which worksto fulfiH this 
purpose. Child advocacy groups are usually organized to collect information 
about and act on behalf of individual children and groups of children in 
neighborhoods and communities — whether at home, at school, or at other 
agencies. One of the activities of ad vocacy is the review and monitoring of 
services that children receive from public agencies such as schools, health 
departments, courts and social services as well as private agencies i.ind 
institutions. This manual provides procedures for this monitoring process. 

Accoimtahility and Child Advocacy 

A basic position of child advocacy is that community agencies and 
orgjmizations should be accountable for the services they provide children . 
Accountability should, by definition, be included in the service mandate 
of agencies. One function of advocacy in a community is to encourage and 
insist on this accoumability. Accountahility is defined here as holding public 
and private agencies responsible for (a) fulfilling the legal mandates under 
which the agency was chartered, incorporated, or funded, (b) using resources 
appropriately and efficiently in providing services for children, and (c) 
providing relevant and effecti ve services wh ich meet the needs and respect 
the rights of children in the community. 

In recent years, there has been a breakdown in the systems which are to 
serve people. Many people needing services are missed, ignored, or denied. 
In addition, the quality of services in terms of cost, availability, 
appropriateness, and effectiveness has been frequently inferior. One reason 
for this breakdown is the lack of information available to agencies and 
consumers about service quality. Many agencies resist needed change and do 
not value or feel a need for such information . Consequently, consumers and 
funding sources have developed increasing interest in accountability, 
evidenced by the increase in requirements for evaluation of services. Many 
funding sources, particulariy the federal government, have insisted that 
programs provide documentation of the delivery and results of services. 

Too frequently , children Co not receive the services they need. One reason 
is because children as a consumer group are vulnerable and poweriess. 
Thus, there is a need to advocate for children by monitoring the programs and 
agencies serving them. A primary concern of advocacy is ensuring that 
,9^ agencies are accountable for the services they actually provide. Accountability 



is a way to require thai-agencies organized to provide services for children 
arc responsive to needs and responsible in their services. 

Levels of Concern in Accountahility 

Concern about the services for children in a community can be at one of 

three levels: 

(1) the entire community (e.g. neighborhood) or group of agencies 
serving children. 

(2) an individual agency within the community, or 
0) an individual child being served. 

These levelsan Uheir relationships are illustrated in Figure I . If your 
interest is in the amount of unmet need for all children, you focus at the 
community levelj.e., the networkof services for children. If you are interested 
in the reduction of unmel needs for children served by a particular agency, 
you focus on that agency and its clients. If you are interested in the 
effectiveness of services for an individual chil<U you look specifically at the 
unmet needs for that child. Of course, problems experienced by a single child 
may uncovera general problem foran agency ora network of community 
services and correction of a problem in an agency will have specific results 
for a single child. In short, the levels are interrelated. Monitoring at the 
community level or individual agency level are described in this manual. 
Individual child monitoring is described in To Protect ami Respect. 

A comnmnitS'level concern relates to all children and the degree to 
which their needs are not met by existing services. Thus, as Figure 1 shows, the 
network or group of services is accountable for their overall (if you will, 
total) effectiveness in reducing unmet needs among children. For example, 
the network of agencies providing medical care and health services for 
children (including hospitals, private physicians, the public health 
department , '>he public schools* etc. ) can be evaluated by the quality of dental 
health, the level of disease resistance, or the death rates among children. 
Coninumity monitoring does not emphasize the clients of one particular 
agency but is concerned with the situation for all children in the community 
or neighborhood. Each single agency within the network may be very 
effective with its own child clients, but the overall effectiveness of the 
network can still be inadequate with respect to the unmet needs of all 
children in the community. 

An ai:enc\*ieyei concern focuses on a single agency or service. It is 
concerned about the effectiveness or success of the agency in reducing the 
unmet needs of the child clients actually served. Agency-level monitct Ingalso 
examines the appropriateness and availability ot the services to the group of 
children for which the service(s) are designed. 

Need to Monitor Existing Agencies 

Determining agency accountability at either level requires two essential steps: 
(1) monitoring, that is, collecting and interpreting information about the 
quantity, appropriateness, costs, and eftectivenessofservices which children 
receive and (2) action, that is, intervening to correct problems and 
inadequacies detected through the review. This guide is to assist child 
^ advocacy groups in carrying out the first step. 
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Figure I. Level of Services for Children 

Through the regular monitoring of child services, problems can be 
identified and agencies held accountable for the services they provide and for 
their use of public and private resources. Effective action is dependent upon 
accurate information about services delivered, child needs met and 
unfulfilled, and child rights respected and violated. 

Who Can Monitor? 

Any citizen in any state has the right to obtain information about agencies 
using public funds. I n many states, private agencies soliciting funds from the 
general public must make certain information available.* An individual, acting 
alone, can monitor a child-serving agency and advocate for the rights of 
children, but realistically both monitoring and action are mor'^ effective when 
conducted by an organized group. Consequently, this guide assumes that 
monitoring will be conducted by an organized group, hereafter referred to as 
an Agency-Monitoring Group (AMG). 

Thisgroupmay be composed of professionals or community volunteers. 
There is considerable merit in having private citizens organized around their 
interest in children. Such groups acting for children can have substantial 
impact on the lives of children an J the institutions and agencies wi^h which 
children interact. 

Although this guide is designed for use by child advocacy groups 
composed of volunteers, the assistance of professionals sympathetic to the 
goals of child advocacy will be most beneficial during Steps III, IV, and VI as 
described later. 

*Check the laws and regutations^conceming private, fi>nd*raising agencies in your stateiQ 0 0 1 S 



HOW CAN YOU MONITOR SERVICES FOR 
CHILDREN? 

Planning And Monitoring 

Monitoring child service agencies is serious business. It requires careful 
planning of objectives, of process, of resources available and of time. 

No matter the size of the community in which an AMG is under way or 
the number of volunteers or paid workers to conduct agency monitoring, the 
following observations are likely to be relevant: 

The child advocacy group has particular interests orconcems which will and should 

be reflected in the monitoring of agencies. 

The number of agencies providing services to children is large.^. 

The process of monitoring lakes time no matter how few agencies are actually 

monitored, or the frequency with which they are monitored. 

The number of people and their available lime for monitoring will be relatively small 

compared to the maximum time needed. 

The scope of monitoring activities must be appropriate to the hours people can 
realistically devote to them. 

Therefore, a number of decisions must be made and problems considered . 
before the final decision to go ahead witli agency monitoring is made. In short, 
the AMG itself must set a boundary or focus of agency monitoring which 
reflects AMG interests, must identify the process or steps necessary for 
monitoring, and must determine what resources are available and required for 
monitoring. 

The Focus of Monitoring 

The focus of monitoring should be adapted to the unique interests of the 
child advocacy group in terms of the conditions and needs of the community 
and of children as perceived by the group. Therefore, the purpose and desired 
results (end products) must be specified from the beginning. 

Two basic decisions are necessary for determining the focus of the AMG 
for monitoring: 

1 . Examine the particular interests and motivations of the group. Agency 
monitoring can be based on concern about a particular agency , such as the 
public schools; on particular needs of children, such as health care; on 
particular services for children, such as custodial care. I n addition, it is very 
important that the group search its reasons, aims, and expectations for 
monitoring. I s it to provide a basis for advocacy action? I s it to providea 
service inventory? Is it for some other reason(s)? Whatever the purpose, the 
AMGmembers should be clear about their intentions when engaging agency 
personnel, parents, media representatives, or others. 

2. Decide on the desired results of agency monitoring. A review of 
possible end products and ways to use information collected may help the 
AMG decide whether to engage in monitoring or what to do with the results 
of monitoring. These are more thoroughly outlined in the final section of 
this guide. 

00 014 
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Some possible results of monitoring are: 

Increased Agency Accountability— holding agencies responsible for the lypes^ 
quality . and etTect of child services, and the openness and appropriateness of the 
^ organizationdeliveringservice^throughacourt suit orexposure through public 
media. 

Identitlcationof Unfilled Needs and Service Gaps — determining where gaps in 
services for children exist and if important needs are overlooked for exposure 
through public media. 

Better Agency Recordsand Self Accountability — stimulating and aidingagencies 
to develop better record systems on the quality and quantity of their services 
and to begin to be (or increase) accountable to themselves via better information. 
A Community Guide to Child Services — publishing a reference book listing by 
type the programs, locations, and eligibility requirements for services for 
children in the community. 

Documentation of the Network of Child Services — describing the collection of 
agencies serving children, how they relate (or fail to relate), and whether they 
function as a system of services to the benefit of children. 

The Process of Monitoring 

The process of monitoring described herein is composed of the six steps 
shown in Figure 2. 

This sequence of steps is designed to systematically accomplish the 
objectives of monitoring, including: 

1. inventorying the current public and private agencies actually serving children 

2. selecting from this inventory those agencies that should be monitored; 

3 . identifying the basic data or indicators which should be collected^ measured , 
or observed during monitoring; 

4. selecting the data collecting technique and collecting data on the selected 
agencies and/or clients served after step 5 is completed; 

5. determining the scope (extensiveness and frequency) of monitoring; 

6. summarizing and analyzing the data collected to determine the need for 
action and to use the data as a basis for advocacy action. 

Each of these six steps outlined above is detailed in the following sections of 
this guide. 

The process must provide considerable latitude and flexibility in 
monitoring according to the focus of the AMG and the scope of monitoring 
as described below. 

The Scope of Monitoring 

Agency monitoring also requires planningand careful attention concerning 
the people who will collect agency information , the time required , and the 
type of information to be giithered. Consequently, making preliminary 
estimates of the time required and actually available by the people involved 
is an important decision. The calculation of a time estimate may seem 
unnecessary , and a ' 'full steam ahead * * approach may appear to be a better 
choice. However, if monitoring requirements exceed resources (people's time 
and interest), the entire process may be aborfed before it can produce the 
desired results. Therefore, this step is necessary to increase the likelihood that 
monitoring can be carried out successfully and that the time of the people 
involved is well-planned and used responsibly. 

00 015 







STEP 1 






Inventory Agencies 








STEP 11 






Select Agencies for 
Monitoring 








STEP 111 






Identify Data to be 
Collected 








STEP IV 






Select Data Collecting 
Techniques 








STEP V 






Determine Scope 








STEP VI 






Summarize Data 







Figure 2. Chart of Steps in Agency Monitoring 



Techniques for making time estimates are outlined in Step V . As a rule 
of thumb , unless the A MG can dedicate an average of five hours per month 
to agency monitoring//? addition to the time required for group meetings and 
other regular work, agency monitoring should not be pursued. 
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STEP I: WHAT COMMUNITY AGENCIES 
SERVE CHILDREN? 

Child Service Agency Inventory 

The first step in monitoring community services to children is to determine 
which agencies are currently providing services for children. This inventory 
will help child advocates become aware of what services are available and 
which agencies are involved, and will enable a systematic approach to be 
used in selecting agencies for monitoring (Step 1 1 in the monitoring process). 
If. however, only the services provided by a specific agency are of interest to 
the group, it is not necessary to compile an inventory. 

Depending on the size of the geographic area of concern (community , 
metropolitan area , or county) the task of compiling a child service agency 
inventory might be difficult. However, if resources are available, an inventory 
is valuable from a number of perspectives. It can become a community 
directory for child services to be used by children, their parents, and 
advocates to select services; it can be used by service agencies to make 
interagency referrals. 

Compilingtheinventory involves two basic tasks: (1) identification of 
agencies or organizations which provide one or more services to children, and 
(2) classification of those agencies by services to increase the usefulness of the 
inventory. 

If a child service directory is already available, it would he wasteful to 
repeat the work. Contact two or three agencies which work extensively with . 
children (e.g. * the public schools or public health department) to see if such 
a directory exists. Ifit does, this section should be studied todetermine if the 
directory meets the needs of the monitoring process outlined herein, and if 
the directory was recently published or updated and revised. 

Identification of Child Service Agencies 
First of all, it is necessary todefine the boundary of such an inventory. For 
the purposes of this guide, theChild Service Agency Inventory is limited 
primarily to those agencies and organizations which provide direct services to 
children, aged 0 to 18, within a given geographic area such as a 
neighborhood, community, city, or county.* 

Direct services are those that focus on thechild. For example, included 
would be child psychological services in which direct counseling and 
assessment were provided for the child, perhaps along with counseling for his 
parents. If, however, an adult sought parental counseling about a child in 
which the child was not directly served, this would not constitute a child 
service. Similarly , a parent may seek public housing, and even though this 
directly benefits his child, the service was not sought specifically for the child. 
If the child is an orphan, the housing ofthat child in an orphanage or foster 
home does constitute a direct service. 

Also excluded from the inventory are those indirect services provided 
by one agency toanother, such as case consultation, even though it may be in 
the interest of a particular child. 0^01? 

♦This boundary is recommended for the purpose of hmitmg the volume of information for the inventory. If desired. 
mdircctj»ervfccs. such as case consultation, also can be fncliided. Such an fnventory would then be morcbeneficial to agencies. 



With these limitations in mind* there are a number of information sources 
that can be used to identify agencies providing services to children. These 
sources include: telephone directories (county and municipal listings and the 
yellow pages), city directories, existing community service directories or 
information councils, the United Fund and similar organizations which 
promote services for children, personal experience of members in the child 
advocacy group, and significant community leaders and primary child 
agencies (such as the public schools' administrative office). Using these 
sources, a list should be compiled of all agencies serving children. 

The kinds of information that are desirable about each agency include: 

1. The lype(s) of services the agency provides for children, 

2. The child population the agency serves and any service limitations by physical, 
mental, socio-economic, or other characteristics, 

3. The primary "sources of funding for the agency (private, municipal-county, 
regional, state, or federal), and 

4. A brief description and name of each program offered for children. 

An inventory of child services must include the service each agency 
provides. Therefore, using Form AMG 01 determine either through mail 
questionnaire or by telephone what services are provided. The categories of 
services on Form AMG 0 1 come from the United Way qf America Service 
Identification System (UWASIS) chart of services explained below and 
described in detail in Table 1. The extent to which the other kinds of 
information (2-4 above) are gathered will depend upon the amount of 
available time of the AMG. If you want to publish a directory of child 
services, a more detailed questionnaire should be used, such as shown in 
Appendix A. 

Classification of Child Service Agencies 

After agencies which serve children and youth have been identified, these 
agencies should be classified to facilitate the next step in the monitoring 
process. There are a number of ways by which agencies can be classified: 

• by type of service 

• by characteristics of clients they serve 

• by primary source of funding 

For purposes of this guide , agencies are first classified by the type of 
services they provide. This method enables the user of the inventory to select 
agencies on the basis of interest in a particular type of service such as housing 
or mental health care. 

The UWASIS chart of community service areas shown in Table 1 has 
been selected as the basis for this classification.* This chart, which is used 
nationally for programs supported by Community Chest or United Way , offers 
a comprehensive way of looking at community services and goals for all 
people within that community. While some of the service areas (such as 
employment and income maintenance) may appear most directly applicable to 
adults, it is important to remember that a segment of the youth population 
(particularly ages 16through 18) may well be eligible and in need of such . 
Q services. The chart covers those services which are directly delivered to 

PR \C people and does not cover services to the community* 
£fvv f I ?' A ft A 1 Q 

h r ii miffnrmmin ♦puWi'shcd by Umtcd Way of America, 801 North Fairfax Street. Alexandria. Va. h V V X. 



Child Advocacy Group 

Address 

City. State 



CHILD SERVICE AGENCY IDENTIFICATION FORM 


Agency Name 




Address 




Contact Person 


Phone 


Check the service areas provided by this agency for children: 


1 Employment 


Notes: 


2 income Maintenance 




3 Consumer Protection 




4 Food and Nutrition 




5 Clothing and Apparel 




6 Housing 


- 


7 Transportation 




8 Justice. Safety. Protection 




9 Physical Health Care 




10 Mental Health Care 




1 1 Mental Retardation Care 




12 Rehabilitation 




13 Formal Education 




14 Informal and Supplemental Education 




15 Individual and Family Life 




16 Personal and Social Development 
and Adjustment 




17 Cultural and Spiritual 

Development and Adjustment 
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TABLE I 

United Way Of America Service Identiflcation System 
Chart Of Community Goals And Service Areas 

GOAL I. ADEQUATE INCOME AND EMPLOYMENT OPPORTUNITY 
Service Area 1. Employment 

Services aimed at creating opportunities for full employment of all those who can be 
g;iinfully employed and can participate in the nation's economy. Services included in 
the system are: manpower development, training, placement and referral: and special 
employment services for those individuals who are handicapped because of some 
social, economic, political or health condition. 

Service Area 2. Income Maintenance 

Services aimed at providing the minimum necessary income to meet the basic material 
needs of those individuals who are unable to provide for themselves. The services 
embrace those using the insurance principle as well as those involving cash financial 
aid to eligible individuals. 

Service Area 3. Consumer Protection 

Services aimed at helping consumers to obtain the optimal return on their income by 
providing protection against fraudulent and unfair trade practices and against 
distribution of products unsafe for human consumption. The system includes services 
which are primarily of an educational or informational nature and also services 
which are of a remedial nature. 

GOAL IL OPTIMAL ENVIRONMENTAL CONDITIONS AND PROVISION OF BASIC 
MATERIAL NEEDS 

Service Area 4. Food and Nutrition 

Services aimed at providing food, food products, and nutrition guidance to the needy 
who are unable or not in a position to provide for themselves. 

Service Area 5, Clothing 

Services aimed at assisting individuals in need of clothing and unable to purchase 
clothing in the open market. 

Service Area 6. Housing Assistance 

Services aimed at providing individuals and organizations with a wide variety of 
assistance related to the housing needs of individuals, families and communities. 
These include: urban renewal and redevelopment: loan assistance for home buying, 
building, repairing and improving, rent and interest subsidies, various forms of 
mortgage insurance: public housing; rural housing; cooperative and multi-family 
dwellings; model cities and new communities; and speciul groups housing, such as 
housing for the aged, Indian housing, veterans housing and college housing. These 
services have come to be almost entirely a public sector responsibility. 

Service Area 7. Transportation 

Services aimed at providing fast, safe, efficient, and convenient means of 
transporting individuals and goodr> from one place to another. These services embrace 
the development, control and safety of every means of transportation for individuals 
and goods, including special transportation needs of selected groups. 

Service Area 8. Justice* Safety* Protection 

Services aimed at the promotion and preservation of conditions that enable 
individuals to live in an environment where they feel safe and are protected from 
anti-social or criminal elements and man-induced or natural disasters, and where they 
can expect to receive legal justice and equal protection of law, and resolve disputes 
without recourse to force. 

GOAL III. OPTIMAL HEALTH 

Service Area 9. Health 

Services designed to foster the health of the individual and to assure care for the 
treatment df illness and disability. These include a comprehensive range of health and 
V ^ health-related services available and accessible to all and providing a continuum of 



care quantitatively and qualitatively adequate to meet thr? health need^ of the 
community. 

Sen lee Arcu 10. MetttuI Health 

Services de^>igned to fostei the mental health ot the indi\idu«d and to assure care tor 
the treatment of mental illness and disability. 

Sen ke Area 11. Mental Retardation 

SeiAices aimed at individuals who. as a result of madequately developed intelligence, 
aie significantly impaired in their ability to learn and to adapt to the demands of 
society. Services range fiom attempts at early diagnosis and prevention to 
comprehensive programs designed to meet the daily needs of menial retardates 
throughout their lifetimes. These .u'e tailored to meet the needs of the retardates at 
various degrees of letardation. such as mild, moderate, severe, and piofound. 

Service Area 12. Rehabilitation 

Services aimed at the physical restoration of disabled persons. The major objective 
is to reduce the handicapping etTect of the disability or to facilitate lealization of 
the disabled person's maximum potential in terms of the disability incurred. 

GOAL IV. ADEQUATE KNOWLEDGE AND SKILLS 

Service Area 13. Education, Formal 

Serv ices aimed at enabling individuals to learn to acquire knowledge and skills 
within a structured framework of institutions (schools, colleges. univer.sities. technical 
institutes, etc.) having well-defined curricula and goals and awarding certificates and 
diplomas u pon successful completion of specified courses of instruction. These services 
cover a formal educational process that ranges from services provided by nursery 
schools to those provided by universities. 

Ser\ice Area 14. Education, Supplementary 

Services aimed at enabling individuals to acquire knowledge and skills outside the 
framework of the formal educational system. These services are geared to provide 
educational opportunities for informal self-teaching, supplementary education, special 
education of the gifted, and compensatory education to fill gaps in the education of 
the disadvantaged. 

GOAL V. OPTIMAL PERSONAL AND SOCIAL ADJUSTMENT AND DEVELOPMENT 

Service Area 15. Individual and Family Life Service 

Services aimed at the preservation and strengthening of families; substitutes for 
family life; crisis intervention and protective services; and certain supportive services 
to individuals. 

Service Area 16. Social Adjustment - 

Services aimed at promoting sound personality development and social maturation of 
individuals through various recreational, group and intergroup relations and activities. 

Service Area 17. Cultural and Spiritual Adjustment 

Services aimed at the promotion of arts and humanities and othei services which 
help human, creativity to flourish and which uplift the human spirit. 



U sing the forms (i.e., AMG 01)on which individual iigency information 
was gathered, identify all those which provide Service Area 1 , ^'Employment/' 
and alphabetize them by agency name to compile a directory. Follow this 
process for each service area. Remember that an agency which provides more 
than one service should be shown under all appropriate service areas. When 
this procedure is completed for all service areas and compiled in order, Section 
1 of the Child Service Agency Inventory is complete (See Figure 3). 

If information is available on the client characteristics and primary 
source of funding, agencies can be further described in detail using the 
information from the form shown in Appendix A. Section II can then be 
completed as shown in Figure 3. 



Child 
Service 
Agency 
Inventory 

Date 




Section I 

Classification 
by 
Service 
Areas 



Section II 

Agency 
Descriptions 



Fiuare 3. Compilation of the Child Service Agency Inventory 



STEP II: HOW DO YOU SELECT AGENCIES 
TO MONITOR? 

Agency Selection 

The purpose of this section is to describe a procedure for determining the 
types and number of agencies to be monitored. 

Considerations in Making Choices 

A suggested procedure for selecting agencies to be monitored is described 
below. First, some considerations which may influence the selections are: 

a) Characteristics of children — the AMG may have a particular concern or 
interest which is used in selecting agencies for monitoring, such as: 

• children with special needs due to phy sical , social , or economic characteristics 
(e.g., physically handicapped, mentally retarded, blind, children from low 
income families.) 

• a particular age group, e.g., preschoolers (0-5). 

b) Sources of funding — the AMG may be most interested in agencies receiving 
primarily public (tax revenue) funding or those using private funding such as 
contributions or United Fund receipts. 

c) Types of services — the 17 types of services used in the inventory section 
represent a convenient way to summarize services. The AMG may wish to choose 
agencies based upon the types of services (e.^., personal and social development, 
health, formal education) of particular interest. 

d) A particular agency* — the AMG may be interested in monitoring one particular 
agency and the array of services or selected services it provides (e.f^. , the public schools). 

Procedure for Selecting Agencies 

The procedure used for selecting agencies depends on the particular focus or 
interest of the AMG. The following suggestions are given for an AMG 
interested in any one of the four possibilities listed above. 

a) Characteristics of children: 

An AMG interestedinaparticulargroup of children, whether definedby 
special needs or age, should first be clear about exactly how they define the 
group of children. If a particular age group of children is of interest, e.f^. , 0-.^ years, 
then the task is simple. 

However, if the AMG is interested in a group of children who have special 
needs, then the task may be more difficult. The AMG should try to clearly 
define what they mean by ''group of children with special needs.'' For example, 
consider the group of children labeled mentally retarded. How does the AMG 
define this group? Do they include all children labeled mentally retarded by 
various service agencies in thecommunity? Do they use thedefinition cited by 
the American Association on Mental Deficiency? Are they only concerned with 
children labeled retarded as a result of brain damage? 

Thereare two main reasons for being clear about what children belong in the 
group. First, clear definition is needed in order to help in the selection of agencies. 
Second . it helps in communicating with the agencies selected. In the first case, 
it would be a waste of time to set up monitoring procedures for an agency that 
had no legal mandate to serve, and therefore no service for, the children of 
interest to the AMG. In the second case, clear definition helps you explain to 
the agency exactly wh^t gro^p.of children you are talking about. 

* >^ ' 0 A 0 0 

•An AMG interested in monitoring of this type can sWp Step 1 1 and turn immediately 18. •/ u Q 
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Once careful definition is reached, the task is to identify the services 
needed by the group ot'children. and the agencies providing these. Most AMG s 
interested in a particular group ofchildren already have good ideas about the 
kinds oFservices needed by these children. Services available in the community 
can be identified by examining the Child Service Agency Inventory. If the AMG 
members are concerned that their knowledge of services is too gcneniU they 
could seek help from professionals who work closely with the group of 
children, to identify the range of services required. 

b) Sources of funding: 

An AMG concerned mainly with agencies supported by a particular type of 
funding could identify the agencies through the Child Service Agency 
Inventory (assuming it was compiled using the form shown in Appendix A). 
There are easy ways to identify agencies by source of funding rather than going 
through Step I . A list of agencies supported by U nilcd Fund is available from 
the United Fund Office. Federal, state or locally funded agencies are usually 
listed in the telephone direcloiy. 

The difficulty arises with agencies supported by funds from a variety of 
sources. If source of support for certain agencies is not easily identified, then the 
agencies should be contacted for the purpo>e of securing the necessary 
information. 

c) Types of service: 

An AMG interested in certain types of service can use one of two different 
procedures for identifying the agencies they wish to monitor. First , if the particular 
types of service have already been identified through group decision, then 
agencies providing these services can be identified through examination of the 
Child Service Agency jnventory. 

On the other hand, if an AMG has not yet decided on the types of services or 
agencies it wants to monitor, the following procedure can be used. It 
involves these three steps: 

1 . Use the U WASI S chart (Table I ) to note the fivegeneral goals and each 
area of service under each goal. 

2. Select the particular goal(s) from the list in which the AMG is most 
interested. CAUTION— selecting all five may be ^'hazardous to your 
health'' because ofthe requirements for monitoring. Then, select oneor more 
of the service areas under each chosen goal you want to monitor. 

3. For each sjiiecific service chosen, identify the agencies by name (and, 
where appropriate, the specific program with in an agency) that provide these 
services, using the inventory developed in Step 1. 

An example will serve to illustrate the three tasks. Table 2 lists each of the 
five goals and corresponding service areas contained on the U WASIS chart. 
On the right side are examples of programs or agencies which correspond to 
the service areas. For example, if Goal I , * * Adequate income and employment 
opportunity ," and Goal IV, "Adequate knowledge and skills," were selected 
because of particular importance to the AMG, then service areas r-3 
(employment , income maintenance, and consumer protection and safety) and 
areas 13-14 (formal education, and informal and supplementary education) are 
the relevant service areas, A review of the goals completes task 1. The 
selection of goals and corresponding service areas completes task 2. 

Task 3 . to identify the agencies providing these service areas from the 
inventory, is also illustrated by Table 2. In this example, **Employment 
Services" are provided by the Employment Security Commission through job 
^ counseling to young people. For example, a young person (say age 17) who 
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TABLE 2 



Example Of Identification Of Agencies To Be Monitored Via Relevant Community 
Goals And Services 





KNamplcN of Agencies Serving Children 
and Relevant Prognims 


(JOAL I. ADKQtArE INCOME AN!) 

EMPLOYMKNT OPPORTUNITY 
Service Area 1. Emplo>ment 

Service Area 2. Income Maintenance 

Scr\ice Area3. Consumer Protection 


Employment Security Commission (Job 

Counseling) 
Employment Security Commission 

(Unemployment Benefits) 
Hotline (Consumer Complaints and 

All vtf*o) 


COAL II. OPTIMAL ENVIRONMENTAL 
CONDITIONS AND PROVISION 
OF BASIC MATERIAL NEEDS 
Ser\ice Area 4. Food and Nutrition 

Service Area 5. Clothing and Apparel 
Service Area 6* Housing 
Service Area 7* Transportation 
Scr\ice Area K. Public Protection, 
Justice, and Safet> 


City Schools (National School lunch 

and Milk Programs) 
Rescue Mission 

Dept. of Social §>m'ices (Foster Home) 
City-County Schools (School Busing) 
Court (Juvenile Correctional Program) 


COAL IIL OPTIMAL HEALTH 
Service Area 9* Physical Health, 

Maintenance and Care 

Sen ice Area 10. Mental HeaUli 

Maintenance and Care 

k»V» * IVV nV^A l\dWI tMf null 

Ser\ice Area 12, Rehabilitation 


County Health Department (Birth 
Control and Family Planning, Well 
Baby Clinic) 

Area Mental Health Center (Child 
Counseling) 

State Center for the Retarded 

Division of Vocational Rehabilitation 


COAL IV. ADEQUATE KNOWLEDGE 
AND SKILLS 
Service Area 13, Formal Education 

Ser\ice Area 14. Informal and 
Supplementary 
Education 


City and Cotmty Schools, Technical 
Institutes, Community Colleges, 
and Universities 

Agriculture Extension Service (home 
economics programs) 


GOAL V. OPTiMAL PERSONAL AND 

SOCIAL ADJUSTMENT AND 
DEVELOPMENT 
Service Area 15. Individual and Family 
Life 

Service Area 16. Social Adjustment, 
Development and 
Usefulness 

?>er%ice Area 17. Cultural and Spiritual 
Enrichment and 
Development 


Family Service (Family Counseling) 

Girl Scouts 
4 H Club 

Little Theaters Churches 
Ballet Schools 
Little Leagues 
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has been employed for a time and is laid off from work is eligible for 
unemployment compensation; then "income maintenance service*' may be a 
relevant service provided by the state unemployment insunmce program. The 
local hotline program might act asa **consumer protection and safely service** 
through referrals to medical services for drug abuse or pregnancy testing or 
through identifying counseling services a young person is eligible to receive. 
**Formal education services** are provided by city, county, and private 
schools. Agricultural extension programs for home economics training could 
be an example of **informal education** in many communities. The list of 
agencies or programs shown in Table 2 is not inclusive but is illustrative of the 
ways agencies can be matched with service types to set a boundary for 
monitoring. 

To list the agencies which are to be included in monitoring use Form 
AMG02. Designate the goal(s)of particular interest.and.usingthe inventory 
of child services from your community, list the agency(ies) or prognimCs) 
which correspond to the goal(s) you have selected. 



AMG02 

WORK CHART FOR DESIGNATING AGENCIES FOR MONITORING 



I) Check goals to be used in monitoring 



□ Goal I 

Adequate income and employment 
opportunity 

□ Goal II 

Optimal environmental conditions and 
provision of basic material needs 

□ Goal III 

Optimal health 

□ Goal IV 

Adequate knowledge and skills 

□ Goal V 

Optimal personal and social adjustment 
and development 



2) For each goal checked, identify from 
the child service inventory in your 
community those agencies providing 
services in the area(s) of interest. 
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STEP IIL WHAT DO YOU WANT TO KNOW 
ABOUT AN AGENCY? 

Identification of Indicators 

Once the agencies to be monitored are determined, the AMG should identify 
the specific information to be collected about those agencies. The purpose of this 
section is to describe three types of agency information which can be obtained 
and to review data collection forms which could be used for each type. 

Types of Agency Information 

Selectivity must be exercised in monitoring agencies, because there is 
insufficient time and resources to collect and interpret all the possible pieces 
of information. Three types of information which reflect aspects o. .Wild 
service agencies and which can be the basis of monitoring are ou tlined below 
and illustrated in Figure 4. 

• Agency/service program management — activities concerned with the overall 

organization of the agency or program, including planning, setting policy and 
procedures, resource allocation, and evaluation and modiflcation.^ 

• Direct service— activities cDncemed with the direct provision or delivery of 

services to children. 

• Service effectiveness— the impact or consequences for children resulting from 

receiving services, particularly the extent to which their problems are solved 
and their needs are satisfied. 

Each of these types of agency information is described in more detail below. 
Suggested indicators and possible data collection forms also are included. 

Selection of Indicators for Monitoring 
An indicator is a specific observation, measure , or reading which gives you 
information about what is going on. The speedometer on your automobile 
gives a reading on OMe indicator of speed, that is, miles per hour. The 
thermometer at your back door tells you the level of the temperature on the 
outside (at least, at your back door). Miles per hour and the height of the 
column of mercury in the thermometer are indicators of speed and 
temperature, respectively. 

If you want to monitor a counseling program for school dropouts, a 
number of indicators may be chosen. For example, if you want indicators of 
direct service, the number of young people served and the average time 
spent per youth in counseling might be used. If you want indicators of 
effectiveness, you might use the number of dropouts who return to school, 
the number who enter other training or find employment, or the number in 
general reporting finding a more satisfying situation, whether school, work, 
military , etc., as a result of counseling. Indicators of program management 
might be whether clients are asked to evaluate the program or participate 
in planning, or if goals for the program have been set, and if the effectiveness 
1^ of the program is assessed by the program administrator. 
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Figure 4. Types of Information Which Can Be Collected for Monitoring 



Recommendations 

The results of effectiveness of a service are, in the final analysis, the most 
important data that monitoring can collect. In other words, services can be 
delivered in the most professional manner with high quality facilities and 
equipment at a low cost. H ow^^ver, this means little if the results of these 
services for children are.not as desired. 

If more information can be collected, then information about direct 
services should be included along with results, since the quality of services is 
directly experienced by the child client and is usually related to effect. If the 
quality of service is poor, then agency management activities of planning and 
evaluation are not successful. 

The entire picture for agency monitoring is complete if indicators about 
management activities can be included. In fact, if problems are detected, 
monitoring management may suggest points of intervention and advocacy to 
correct and improve services. 

The stated purpose of this section is to aid in the determination of the 
type of information to be collected. 1 1 is recommended that if only limited 
information can be collected, information on service effectiveness be given 
priority.* 

A checklist for identification of indicators is provided toassist in this 
planning step. See Form AMG 03. 0 0 02 9 
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♦The decisions concerning selection of the type of agency information and appropnatc indicators for date collection must be 
closely related to the scope of the AMG monitoring project. Scope is discussed in Step V. 
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Monitoring Service Effectiveness 

The ejfci i ofa ser\ ice refers to the amount of need reduction or level of need 
satisfaction w hich occurs for a child client due to participation in the service. 
The effect which is a result or outcome ofa service is not the same as the 
provisionof the service, however, as illustrated by the following examples: 

Service Effect 

dental care sound and corrected teeth 

eye examination and corrected vision 

glasses fitted 

classroom instruction improved reading skills 

in reading 

job counseling satisfactory employment 

Few agencies obtain follow-up information on the effects of their 
serv ices: therefore, such information is not readily available. Most frequently, 
agencies report the quantity of direct services delivered .i.e., number of clients 
served.numberof materials disirbu ted. number of dollars spent, etc. Here, 
howev er. w e are concerned only with results achieved and the degree of need 
reduction clients experience due to the services. 

Effectiveness can therefore bedefined as the degree to which unmet needs 
are reduced for a child or group of children as a result of receiving direct 
serv ice. or for a community as a result of having services available. Depending 
upon the monitoring extensiveness chosen (see Step V on Scope), various 
indicators can be selected for monitoring. 

The first task is to select the effectiveness level of interest . The second 
task is to define the specific effect you want to monitor and select the 
indicator(s) or means to measure or observe that effect. 

Level of Concern. As illustrated in Figure 1 and Table 3, service 
effectiveness can be assessed at the community level, agency level, or 
individual child level. The focus of this manual is on the community and 
agency levels. 

Basically the difference between the community and agency levels of 
effectiv eness is the population of concern, i.e. , "all children in the community" 
versus "the clients ofa particular agency." 

If your concern is primarily with theoverallcommunity and the extent 
of unmet need in a certain goal or sei vice area, the service effectiveness of all 
agencies relevant to that goal or service area (see the Ch ild Service Agency 
Inventory) can be determined by a community audit.'- 

Desired Effects. The desired effects must be stated by the A MG depending 
on what level is to be monitored. At the community level, a desired effect must 
be stated as a condition of all child community members . If. on the other hand, 
the service effectiveness for the clients of a particularagency is of concern^ 
the desired effect that is defined must be relevant to the purpose of that 
agency and only clients of that agency can be assessed. Whether an agency is 
responsive to all clients who appear for service is an indicator of direct service, 
not effectiveness. If you are concerned about children who should be 
served but are missed, then the focus of effectiveness is at the community level. 

^Aionmttimtsaudtt is a means of determiningthe level of unmet need in the community. It may be conducted thruugn a survey 
ofa sample of children who are believed lo be representative of the entire community. (See Appendix B on Sampling 
Procedure. 00 030 
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CHECKLIST FOR IDENTIFICATION OF INDICATORS FOR MONITORING 

□ Service Effectiveness 

□ Direct Service 

□ Management 



1) Check the type(s) of agency 
information selected for 
monitoring: 



2) If service effectiveness: 

a) Select the effectiveness level 
of concern or interest: 

b) Define the desired effect(s) 
you want to monitor: 

c) Select appropriate indicator 
for each effect: 

d) Identify technique for data 
collection of each indicator: 

e) Identify source of data for 
each indicator: 



□ Community Level 

□ Agency Level 

□ Form AMG 04 Completed 

□ Completed (Form AMG 04) 

□ Completed 

□ Completed 



3) If direct service: 

a) Select the aspect of direct 
services to be monitored: 



□ Service Requirements 

□ Service-Community Relationships 

□ Service Activities 



b) Complete the tasks outlined 
for each on Direct Service 
checklist: 

4) If management: 

a) Select the aspect(s) of 

management to be monitored: 



□ Completed 



□ Planning 

□ Policy and Procedures 

□ Resource Allocation 

□ Evaluation and Modification 



b) Select the technique(s) for 
collecting data: 

c) Collect data using suggested 
questions as outlined plus 
others as desired: 



□ Completed 



□ Completed 
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TABLE 3 

Levels Of Service EfTectiveness With Examples Of Desired Effects And Indicators 



SERVICE EFFECTIVENESS 



Le\el of 
Concern 


Desired Effects 


Types of Indicators 


COMMUNITY 


A <*(in(lifi(in (li^wirpfl \t\r 'ill 

/\ V^MIUIll^Ml UvMlwU 1^11 (III 

children in the comfnunity, 
Mich as **high disease 
resistance against measles 

f)f :ilt I'hildrpn m (*(immiin!f 

Kfl (HI WIIIIUIWll 111 V^MIIIIIl.11111^ 

B • 


i^u. cases ui lucasics in 
community during 
the year 


AGENCY 


Specific agency goals such 
as "Increase the reading 
level of all children by 
one grade in a year." 


Scores on Reading 
achievement tests of 
all public school children 
during the given year 


INDIVIDUAL 
CHILD 


Maintain adequate level of healtii lor a child. 


Health status obtained 
from a physical check-up 



Indicators* The type of i ndicator chosen foreither of the above levels is 
dependent upon the requirements for collecting such information. For some 
indicators a personal interview is the only way to get information: for others, 
going through agency records is the only source of the indicators . Examples 
of community and agency level indicators of service effectiveness are shown in 
Table 4 organized by U WAS IS goals. 

Two possible approaches to determining service effectiveness are (I) 
obtaining through interviews the level of need satisfaction forchildren , and 
(2) obtaining from generally accepted tests or indicators the levels of need 
satisfaction of agency clients or children in the community.;^ 

The first approach requires contacting the parents, guardians, or individual 
child advocate, or even the child , if he or she is old enough , to determine his 
need satisfaction level. Using mail, telephone, or personal interviews, a 
sampleof the entire comntunity era sample of clients of a particular agency 
can be contacted concerning the stated desired effects. 

Forexamplejf thedesired effect were for "all children between theages 
of 16 and 1 8 who are currently enrolled in school to have summer employment 
if desired/' then indicators might be: 

(a) number of children who desire a summer job for the summer of 1975. 

(b) number of children who had summer employment during 1975. 

Corresponding questions for an interview with a sample of teenagers 16 to 18 
might be: 

1. Are you currently enrolled in school? 

2. Did you seek employment for this past summer? 

3. Were you employed during the summer? 

'A note ofcaution about indicators of service effectiveness is needed. Effectiveness has beengeneralty ignored oroverlooked 
^ in service agencies. Correspondin^y* few good indicators of effectiveness exist. Few standard* general! v accepted tests are 

1^ r%Y/^' available. Furthermore, many existing standardized test results such as IQ scores are very questionable as indicators, 

p K I ( particularly since standard tests often have strong cultural biases. While the emphasis here on service effectiveness is 

L,„.„,.,j^^,„^ important, you should be aware of the dilHcQltie^ llle^ii^an^or developing good indicators of effectiveness. 



TABLE 4 

Examples Of Indicators Of Service EfTectiveness By UWASIS Goals 



GOAL I. ADEQUATE INCOME AND 
EMPLOYMENT OPPORTUNITY 


1. Number of teenagers who obtain summer or 
year-round employment 

2. Number of children covered by health 
insurance 

3. Number of dropouts from school who 
become employed 


GOAL 11. OPTIMAL ENVIRONMENTAL 
CONDITIONS AND PROVISION OF BASIC 
NEEDS 


1. Number of children having **adequate" 
diet 

2. Number of children living in '*standard" 
housing 

3. Number of children who are abused or 
neglected 

4. Number of children in trouble with law 


GOAL III. OPTIMAL HEALTH 


1. Number of children with uncorrected 
vision, dental* hearing, orthopedic, 

or other "correctable" health problems 

2. Number of child suicides or attempts 

3. Number ot children with VD 

4. Number of problem pregnancies among 
children 

5. Number of children involved in serious 
injuries due to auto or hazardous 
facilities 

6. Number of child "problem-days" due 
to illness 

7. Number of child deaths by age 

8. Number of drug-related problems 


GOAL IV. APPROPRIATE KNOWLEDGE 
AND SKILLS 


1. Percent of all children achieving below 
their ability 

2. Number of "dropouts" 

3. Number of high school graduates seeking 
work who become employed 

4. Number of children referred to special 
classes due to failure or under achieve- 
ment in regular classes 

5. Number of children needing compensatory 
education 

6. Number of children below appropriate 
reading skill level 


GOAI V OPTIMAI PERSONAL AND 
SOCIAL ADJUSTMENT AND 
DEVELOPMENT 


1 Number of childr£»n ^^eekinc ctilttirnl 
programs (ballet, theater, speech, 
music, etc.) 

?. Percentage of all children participating 
. in organized sports or recreational 
programs 

3. Number of children running away from 
home 

4, Number of children having difTiculties 
relating to peers or being rejected 
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Collecting and summarizing these data for a sample of teenagers from the 
community or from an agency such as a State Employment Commission would 
provide indication of the level of need satisfaction concerning summer 
employment. If the sample were of the entire community, the effectiveness of 
all agencies relevant to this service area is monitored in aggregate. If the 
sample were of clients of the Employment Commission, additional questions 
on the interview might be: 

4. How well did the Employment Commission help you in meeting your need for 
summer employment? 

5. Are you satisfied with the assistance you received from the Employment 
Commission in finding a summer job? 

6. How did you get your summer job? 

Located it myself 

Referred by Employment Commission 

The second approach involves using agency records to determine the level of 
effectiveness of agencies in a community or among the clients of one agency. 
For example, if the desired effect is a **high disease resistance against measles 
among children in Community X,'' then indicators may be 

1 ) number of child cases of measles in ih e community in the last five years by 
year. 

2) number of children who have received immunization against measles during 
each of the past five years. 

The Public Health Department maintains public records of this type. Another 
source of data might be public school health records for school aged children. 

These data can be used to monitor the effectiveness of the Public Health 
Department only (using only a sample of their clients during a particular time 
period) or data can be collected to indicate level of disease resistance against 
measles among the entire community (using a sample of children from the 
community.) 

Another approach to determining service effectiveness involvesa study of 
public documents, such as census data, for the community level or -eview of 
any reports which agencies may have produced for the agency level. 

Examples of other community and agency level indicators of service 
effectiveness are listed in Table 4, organized by UWASIS goals. 

Summary. If AMG chooses to monitor service effectiveness, the level of 
concern (community oragency)mustbeselected,thedesired effects must be 
stated, and the indicators of those desired effects must be selected. The 
selection of indicators is dependent on the techniques and data sources 
available. 

Planning form AMG 04 can be used to record these decisions. 
Monitoring Direct Services 

Direct service refers to those activitiesactually involved in delivering sei vices 
to clients- Monitoring direct services can involve one or more of the following: 

• Service Requirements 

• Service — Community Relationships 

• Service Activities 
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PLANNING FOR 

MONITORING SERVICE EFFECTIVENESS 



Level of Monitoring: □ Community □ Agency 



(Name of Agency) 



DESIRED EFFECTS 


INDICATOR 


(Define the desired effects you 
wish to monitor below,) 


(Identify the appropriate indicator(s) 
for each effect,) 


Example (Agency: Public Health Department) 
High disease resistance of all children 
against 


of children receiving inoculations 
for each diseas.^;. 


a) smallpox b) typhoid 
c) diptheria d) measles 
e) polio for all children 

I,C^ i\fU/C IllllULUIallUllS* 




1 




2 




3 




4 
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Service Requirements* At the most general level, the legal mandates and 
federal and/or state policy and procedural guidelines can be obtamed for a 
given iigency . Then, the service programs and activities actually provided by 
the agency are matched tothe requirements for service. For example, if a legal 
mandate for the pu blic schools in community X were * *to provide for ^/// 
children within Community X physical training and education daily,'' then it 
should be determined if in fact that service activity was provided by every 
public school daily for all children. 

Service-Community Relationships. This approach is primarily concerned with 
the degree to which appropriate services are available tochildren within a 
community, and the difficulty clients experience in obtaining available 
services. 

I n other words , for the goal and service area(s) of concern , what services 
are actually availablefor children in the community (across several agencies or 
within one agency)? 

Secondly, what is the estimated community need for these services? 
Use public documents (census data) or a survey to estimate need. For 
example, there are 2,096 children in Community X who need day care 
services daily. 

Third, do the available services match the need for services? That is, are 
there spaces for 2 ,096 children in the day care facilities of Community X? 

Fourth, what deters clients from obtaining needed services? The 
costs, eligibility requirements, and proximity of facilities often may prevent 
children from using services. For example, a child may not be able to meet a 
need for group recreation because the closest park is two miles away and he 
has no transportaion. Or a child may not receive needed food in a school lunch 
program because his parent's income level makes him ineligible. 

Use of this approach allows comparison of service availability and 
appropriateness to the needs of community children. The emphasis is on the 
the community need, the amount of services provided by one or more 
agencies to meet that need, and the constraints or deterrents that prevent 
use of the service. 

To organize the process of selecting indicators and collecting data, a sample 
data collection form, A MG 05, and data summary form, AMG 06, are included 
illustrating the above example. 

Service Activities. The third approach to monitoring direct services is 
primarily concerned with details about specific service activities provided by 
an agency or across a number of agencies. For example, if the service area of 
concern were "mental health care," all mental health service activities of 
the county mental health center, of the public schools, of the department of 
social services, and of other agencies which are provided for children may be 
monitored. Or one specific service activity forchildren, such as individual 
therapy, may be of importance and require monitoring of both public and 
private agencies and professional practices in Community X which provide 
that service to children. 

Direct service acti vities are those involved in actually providing a service 
to clients. These activities generally are the most familiar to the public. Due 
to the variety of service activities, a basic grouping is shown with examples. 



DIRECT SERVICES 

SERVICE-COMMUNITY RELATIONSHIPS 
Data Collection Form 



Goal/Service Area of Concern: 
Optimal Personal and Social Development 
(Day Care for Young Children) 



Agency Name: Pam*s Day School 
Agency Address: XXX Street, City Slate 
Contact Person: Pam Baker 
Date of Contact: January 17, 1974 



Services/Programs 
Name 


# Children 
Requesting 
Service in Year 


Capacity: 
Maximum # 
Children 
Who Could 
be Served 
in Last Year 


# Children 
Served in 
Last 
Year 


Eligibility 
Requirements 
or Other 
Constraints 


I/Day Care Centers 


75 


50 


50 


Cost $20 week 
Transportation 
not provided 
Medical Exam 
All Children 
accepted 
Hours: 

7AM to 6 PM 


2/Nursery 


50 


25 


25 


* 


Totals 


125 


75 


75 
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DIRECT SERVICES 

SERVICE*COMMUNITY RELATIONSHIPS 
Data Summary Form 



Goal/Service Area: 

Optimal Personal and Social Development 
(Day Care for Young Children) 



Community Need for Service (estimate) 
20% children per day 


Eligible Children 
1361 


Agencies Who Provide Service 


Capacity 


# Served 


# Requesting Service 


I) Pam's Day School 


75 


75 


125 


2) Motherland 


XX 


XX 


XXX 


3) Play School 


XX 


XX 


XXX 


4) etc. 


X 


X 


XX 


5) etc.. 


X 


X 


XX 


6) etc. 








Total 


1361 


1361 


2096 



Constraints 



Cost 

Transportation 
Eligibility Requirements 
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Intake and assessment activities are those determining the unmet needs of a 
child to which this or other agency can respond. At this point, basic information about 
thcchild is obtained upon which future services depend. I n a child service like public 
schools, these activities would include grading, achievement and other testing, and 
teacher observations. These activities are used to determine the situationand needs 
of a child. 

Service planning* is the development of a schedule for and description of 
services which are believed relevant to the child. This planning may result in the 
decision to prov ide the services b y the agency , or to refer the child to another agency ^ 
or possibly torefuse to provide any service. Service plans may be the result of careful 
evaluation of needs and thoughtful identification of relevant services based on intake 
and assessment procedures. On theotherhand.a very unsystematicand capricious 
useof what is available" may be a basis for decisions with little attention to the 
child. Or children may get the same basic plan unless a problem arises (such as in 
public schools). 

Service delivery activities is the actual delivery of a service to the child*client , 
such as medical treatment , ^special training, inoculation, eye examination and 
fitting of glasses, dental care, classroom instruction or counseling, school lunch 
program, and so on. 

All direct service activities can be classified into one of the above groups^ 
which provides a convenient way of looking at services provided. Many other 
service activity classifications are available for use. 

Indicators. Therearea number of important indicators which are related to 
the actual delivery of a service to children in a community, including the 
following: 

CLIENT-VOLUME AND FLOW 

1) Total number of children accepted for service during a given time period 

2) Total number of child cases closed due to completion of service during same 
time period 

3) Total number of child cases closed due to referral to another service or agency 

4) Total number of child cases closed due to client terminations or **drop outs** 

5) Total number of requests for service who are not accepted 

6) Total number of child cases cScsed mat are followed-up 
SERVICE RESPONSIVENESS 

1) Average length of time from enrollment to completion of service 

2) Average length of time from initial request for service and beginning of 
actual service delivery 

3) Frequency of service delivery to a client (daily, weekly, monthly) 

4) Service capacity vs. service case load: 

How many children (maximum) can be served in one day? 
What is average number of children served in one day? 

5) Are the individual rights of children monitored? 
TYPES OF SERVICE ACTIVITIES 

1) List and description of all service activities and/or programs provided by 
agency 

2) Of these services, which are: 

a) Intake and assessment 

b) Service planning 

c) Service delivery 



TABLE 5 

Suggested Techniques For Collecting Data Concerning Service Activities 



CATEGORY 


SU(iGlvSTKI) TECHNIOl KS 


CLIENT VOLUME AND 
FLOW 


Rich of the indicators included in this catetsory can 
be monitored using either 

(1) agency research evahiation reports 

(2) searching the tiles of all child cases during 
a given period of time 


SERVICE 
RESPONSIVENESS 


Avcr.ige lengths of time and ficquency of service can 
be determined through 

(1) estimation by service staff or 

(2) searching case tiles to identify length 
required for ^ sample of clients, totaled and 
divided by the number of clients in the sample 




Service capacity can be determined by 

(1) ResidentiaMnpatieni service: No. of beds for 
children 

(2) Non*Residential-Outpaticntc total number of statT 
times the maximum number of children one stalT 
person can ser\ i; in a day 

Average ntmiber of childn^n served in one day can be 
determined by 

(1) observing the number of children using the 
service on a given day or 

(2) service stalT estimates 
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service under consideration and may first require 
seaich of legal and departmental policies and 
gtu"delines, The degree to which these rights are 
(1) specitied and (2) communicated to children should 
be determined. (Examples: are formal eligibility 
requirements actually used? Are rights to hearings 
or choice of treatment upheld?) 


TYPES OF 

SERVICE 

ACTIVITIES 

tm •••••• m 


Obtain a list and description of all service 
programs or activities provided by the agency, 
Catt?i*Mri/£r iht^m? st?rvices hv tvnt? 


SERVICE 
RESOURCES 

• 


For each stwicc piogram or activity determine from 
budgets or personnel allocations the total number of 
direct service staff by position and level of 
training and experience. This may require personal 

infi^rvipu/v u/ifh individual vf!ifT iir fht^ Miri*(*f4ir ill 
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the program, 




Determine the total number of children served by 
inai prognim or <iciiviiy lor <i given icngin o; 
time and divide by the number of staff providing 
the service to obtain a ratio of child clients to 
staff. 




Obtain the total budget allocation to each service 
program and/or activity from the director of the 
program or appropriate management. 


SERVICE COSTS 


Cost per unit of service delivered can be estimated 
by a representative of the agency or calculated in 
a number of steps. 



If service is Residential or Inpatient, 

(1) determine the total annual budget for the 
service during the last fiscal year 

(2) determine the total number of clients served 
during the same fiscal year 

(3) divide the total budget by total number clients 
to get cost per client 

(4) determine the average number of days a child stays 
overnight in the facility for service 

(5) divide the cost per client by the average number 
of days per stay to get cost per day 



If service is Non-residential or Outpatient, 

(1) determine the total annual budget for the 
service during the last fiscal year 

(2) determine the total number of child clients 
served during the same year 

(.1) divide the total budget by the total number 
of clients— cost per client 

(4) determine the average number of visits a child 
makes for the service (examples: for public 
education, approximately 180 days per year; 
for counseling, approximately 4 one-hour 
visits). 

(5) divide the cost per client by the average ^ 
number of visits noting the length of one 'visit equals 

I cost per service day or cost per hour 

SERVICE RESOURCES 

1) Total number of direct service staff by position and level of relevant training 
and experience 

2) Ratio of child clients per service staff (# children/# -service staff) 

3) Total budget per year per service activity or program per child 
SERVICE COSTS 

Determine the cost per unit of service delivered: 

1 ) Residential or 1 npatient , that is, total cost per day of care per child or service 

2) Non-residential or Outpatient, that is, either 

a) cost per hour of service 

b) cost per service day 

c) cost per child client 

This approach to monitoring direct services can be the most detailed and 
complex, therefore requiring greater AMG resources. One or more of these 
categories of indicators for monitoring service activities can be selected. 
Table 5 suggests techniques that might be used in collecting the data for each 
category. 

Summary. Monitoring of direct services, then, can take a number of 
approaches. First, AMG should select the approach(es) of interest: Service 
Requirements, Service-Community Relationships,and/or Service Activities. 
Then, the appropriate tasks as outlined in checklist AMG 07 should be 
completed. This type of monitoring may require greater professional expertise 
orconsultation than others, particularly in the selectionand implementation 
of techniques to obtain the needed data for indicators. More specific 
procedures and forms should be designed which are directly related to the 
aspects of direct service AMG desires to monitor. 



Monitoring Agency/Service Program Management 
Management of an agency functions in a controlling and guiding capacity in 
the same way the driver of an automobile steers and controls the vehicle. 
Management decisions are carried out by direct service activities. 

Management for a service program and/or agency provides the overall 
decisions about the content of services provided , provides the policies and 
procedures employed, secures and allocates resources, and is responsible for 
the quality of services and makes necessary modifications. 

Here, management activities have been categorized in order to identify 
points where monitoring may take place. These activitiesare carried out by 
agencies in differing degrees of emphasis and detail. Some management 
decisions (such as program guidelines) may be specified outside the 
organization (by a funding source, for example). 

Planning activities determine the goals and direction of the agency including 
what services will be provided, what resullsare expected, when, why, to whom, 
and what resources are required. 

Decisions about policy and procedures set the operational restrictions, controls 
and specifications for the agency and its services and affect the manner in which 
services are delivered. 

Resoiifce'aflocation «c//i77/VA'"activate services (give the' **go afieaci^'Vand 
determine the way money will be spent for supplies, travel, personnel. The 
selection of personnel as a type of resource allocation can be included here. 
Evaluation and modification activities are those which assess the quality, 
appropriateness, and effectiveness of services provided. If goals are not met, 
then corrections or modifications may be needed. 

Followingare questions for each of these categories that might be included on 
a checklist to review the management activities of an agency . The information 
may be available through attending public meetings, re viewing documents, 
and interviewing staff. 

Planning Activities 

1. Does theagency do any planning forthe services or programs forchildren? 
Are plans written down and available for you to review? 

2. Ifplans are available, are desired effectsor goals forchildren specified in 
terms of the reduction of needs? 

(NOTE: a description of the service or program is not the same as what results the 
service should produce for children. For example^roviding job counseling for 
school dropouts is a service description. A desired effect forthe program could be 
**Sixty percent of youths are either employed or back in school within 90 days 
following counseling.") 

3 . Does the agency have data about the extent of unmet need the program or 
service addresses? That is, does management actually know the needs of the children 
to be served? 

4. Does the community/neighborhood have opportunity to participate in 
planning? How open is the planning process and can the clients establish or aid in 
establishing the desired results of a service? 

5. I s the service plan made public for reviewer approval? If a board reviews and 
approves plans, how well are community members represented by the board? 

6. Who selects the plans for services or programs which will be actually 
implemented? If funding is the critical step, who decides about funding? 
(Note: Staff training should be evaluated not as much by the degrees they have 
obtained but by the appropriateness of training to the services delivered. 
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CHECKLIST OF TASKS FOR MONITORING DIRECT SERVICES 



□ 1) Service Requirements 

□ Task 1) Obtain the legal mandates or federal-state policies 
Complete concerning agency A from the state legislature, etc. 

□ Task 2) Determine what services (programs and/or activities) 
Complete are currently provided for children by Agency A, by 

analysis of material gathered from completing Task 1, 
or other sources. 



□ 2) Service-Community Relationships 



□ Task I) 
Complete 

O Task.2]L 
Complete 

□ Task 3) 
Complete 

□ Task 4) 
Complete 



Given the goal and service area(s) of concern, list 
all services (programs and/or activities) available 
for children. 

Determine the amount of comiTVunity. ne§d .for thesp 
services. 

Determine the maximum and actual amounts of 
available services. 

Determine extent of deterrents which limit service use 
by children. 



□ 3) Service Activities . . 

□ Task 1) Select indicators of mterestfor monitoring. 

Complete 



□ Task 2) 
Complete 

□ Task 3) 
Complete 



Select technique for collecting data. 
Collect data as instructed. 



Evaluation and modification activities 

1 . Docs the management of the agency monitor the quality of services provided 
children (i.e. , how well children are treated, are their privacy orrights respected , are 
legal requirements for services for individual children fulfilled, etc.)? 

2. If the quality of services is below par does management make changes to 
correct the problem(s)? 

3. If so, do such changes make any difference? Are things improved? 

4. Does the management monitor the effectiveness of its services for children 
through follow-up of clients? Is there assessment of client satisfaction? How much 
is known about the consequences for children served by this agency? 

5. If the services are not producing the desired effects for children, does 
management make changes to correct the problem(s)? 

6. If so, do such changes make any difference? 

Monitoring the management of a child*serving agency will require careful 
observation and judgement. You must get the ••feel" of an agency's 
management by attending meetings, talking to staff, looking at reports, 
etc. The suggested checklist of questions above (wh ich can be expanded) will 
provide guidance in monitoring and identifying the types of questions you 
-might ask. ... - .... 

Summary 

Step 1 1 1 has addressed the question: What Do You Want to Know About An 
Agency? by looking at three types of agency information: service 
effectiveness, direct services, and management. For each of these types, 
indicators and checklists or data collection forms have been suggested . The 
decisions related to selection of which type of information and which indicators 
to select are important and require careful consideration. These selections 
are very related to decisions of techniques (discussed in Step I V) and the 
scope and frequency of monitoring (discussed in Step V). 
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STEP IV. HOW DO YOU GO ABOUT 
MONITORING? 

Data Collection 

By the time you reach this step in the guide you have covered: the inventory 
of the ch ild service agencies (Step 1), the selection of agencies to monitor 
(Step 1 1 ), and the decision about the information to be collected (Step 111). 

The purpose of Step IV is to outline the process of actual data collection , 
that is, putting the monitoring process into action. This will be followed by 
Step V , which covers the estimation of time required for monitoring, and Step 
VI, which presents techniques for summarizing and analyzing those data. 

Basically, data collection involves three tasks: 

(1) selecting a data collection technique, 

(2) organizing resources to collect the data, and 

(3) collecting the data. 

Since the organization of resources a'nd procedures for colfecting data * 
depend upon the technique selected , these tasks are described below according 
to the technique. 

Data Collection Techniques 

Five primary data-collection techniques can be identified: 

1. Public documents, meetings, and issue research 

2. Mail surveys 

3. Telephone interviews 

4. Personal interviews 

5. Client follow-up through agency records 

Following are summaries of each of these techniques: 

Public documents and meetings and issues research. This technique involves 
researching information about an agency from public, state, and agency 
libraries , in newspaper and magazine articles , annual agency statistical reports 
for previous years, budgets and service plans for upcoming years, proposals 
for funding, and special evaluation and research reports. Any citizen has a 
legal right to obtain certaiti public documents about any public agency. 
Depending upon state law, such information as total agency budget, 
personnel salaries, and numbers and names of clients served are often 
considered public information. Check the law in your state to establish exactly 
what information is considered public. The kinds of data that can be 
gathered using this technique include annua! budgets, number and 
classifications of children served, and perhaps service goals and program 
descriptions. These data are already in summary form and may only allow 
minimum alteration and interpretation. 

Another aspect of this technique may involve attending public meetings 
to determine current significant issues, the positions and values of agency 
administrators and board members. This approach , however, offers a means 
of monitoring only in it^ simplest form. 
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Mail surveys. A mail survey consists of a questionnaire and a set of 
instructions for its completion designed to obtain certain information which is 
mailed along with a cover letter to a list of agencies or service clients. 

This technique involves activities such as the duplication of a cover 
letter, instructions and questionnaire, obtaining a mailing permit or using 
regular postage, and addressing and stuffing envelopes. Whil the approach 
requires some effort by a number of people du ring a concentrated period of 
time, it requires less overall time than other datacollection techniques and no 
face-to-face contact with the interviewee. The disadvantages include ( 1) the 
time it takes for questionnaires to be completed and returned, (2) the 
possibility of a low percent of return ,* and (3) the possibility of a biased 
response in those that are returned. Ways to increase the percentage of 
return are to enclose a self-addressed , stamped envelope for the convenience 
of the person contacted and to follow up with a second letter or phone 
interview if the questionnaire is not returned within a specified period of time 
(such as one week). 

The problem of alow percentage of return is not, however, as important 
as the passibility of a biased refiponse . For example, if less than a 50% return 
is made, check to see if responses are positive because the clients responding 
received good service and want to report that or are negative because the 
clients are dissatisfied and want to complain. To reduce the possibility of 
bias, a sampling procedure such as those described in Appendix B can be 
used in selecting the service clients or agencies for the mail survey and for 
any personal follow-up to assess bias in responses. 

Telephone interviews. An interview by phpne is a way to question agency 
personnel or clients personally but with less time and difficulty than a face-to- 
face interview. It requires that the AMG representative making the contact 
clearly identify himself or herself, give the reason for the call, and be flexible 
and persistent. Consider both (1) the convenience of the interviewee and (2) 
the need to obtain information. However, the interviewer should ask the 
questions in such a way to avoid prejudicing the interviewee with his or her 
own personal bias. 

This technique allows contacts to be madeconveniently by the AMG 
representative. The reluctance of people toanswer questions, requests to 
call back, and delay in data-gathering can be frustrating. The telephone 
interview may result in more or less information than desired, depending on 
the skill of the interviewers and the degree of cooperation by the persons 
contacted. 

Sampling procedures described in Appendix B can be used in selecting 
clients for interview. 

Personal interviews. A personal interview involves a face-to-face contact 
with an agency representative or client of a service in which an interview form 
is used to obtain the desired information. An interview form is a guide used 
by the interviewer to ask questions. This technique requires (1) a phone call 
to make an appointment for the interview, (2) clear identification of yourself, 
and (3) a statement of the reason for the interview. The success of this 
Q technique is increased by an appropriate and neat appearanceof the AMG 

interviewer and by the manner of questioning used. Often more information 
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can be gained or clarified by tactful probing. It is also important to remember 
that an agency representative has a busy schedule and that you are taking 
the personal time of clients. The interview should be as brief and to the point 
as possible. 

Due to the nature of face-to-face interviewing, this technique offers the 
opportunity to obtain more information and to explore with less structured 
questions a greater range of issues than a mail or phone interview . However, 
the technique requires more time on the part of AMG representatives than 
any other technique. Interviewer bias may also affect the answers. 

One of the sampling procedures should be used in selecting service 
clients to be interviewed. Personal interviewing can increase the 
representativeness of a sample, since the interviewer can ensure that sufficient 
interviews are obtained. 

Client follow-up through agency records. CoUectingdata from agency records 
about clients is a useful technique when monitoring the service delivery of 
an agency or when intensive follow-up of clients is impossible or not desired . 

This technique involves securing the confidence and permission of the 
agency manager, since he must protect the rights of clients to confidentiality . 
The purpose and intent of the data collection must be clearly spelled out to 
increase the likelihood of his cooperation. Once support is obtained, the records 
of the agency can be researched for the desired information in a systematic 
manner. This can be tedious and time-consuming work, but responsibility 
can be shared by a number of AMG representatives. 

If the agency manager is concerned about confidentiality of client 
records, this is a good sign. I n like manner, a child advocate should also be 
concerned. If access to records can be obtained, then the AMG should 
respect the confidentiality of individual information. 
The tasks in researching client records are: 

1) to choose an appropriate sampling procedure, 

2) to design a form for recording the information taken from records, 

3) to locate and record the information. 

Selection of Technique 

Each data collecting technique has related advantages and disadvantages 
which make selection of the appropriate technique a significant decision. 
Recommendations shown in Table 6are listed for each technique according to 
the type of information selected for monitoring in Step III, that is, the type 
of information desired about the agency (management, direct service, or 
effect). 

The final decision about which tech nique(s) to use should also consider 
the amount of time available for and frequency of monitoring, as described 
in Step V. 

Thoughts About the Process of Data Collection 

Data collection may involve either the clients (children and their parents) 

of a service agency or representatives of the service agency. At this point, it is 

relevant to point out some important considerations related to each source of 

information. 



Service clients. To obtain a "representative" response from clients about the 
effect of a service or their satisfaction with a particular service agency, two 
things must be considered. One , unless the client population is small (50 or 
less), a sampling must be used toobtain an accurate representation of clients 
without contactingevery client. Secondly, confidence and trust in the AMG 
must be obtained to avoid biasing the response of the client or his 
representative. A clear statement of the position of "advocacy" and the 
purpose of the information and use of data-collecting techniques which reflect 
a consideration of the client will increase confidence. 

Agency representatives. The intent of the monitoring process, to determine 
theaccountabilityofan agency, may produce some suspicion and resistance 
or hostility on the part of agency managers or administrators. Therefore, the 
manner in which an agency isapproached will havea great effect on the 
success of the data collection process. It is recommended that the AMG 
representative clearly identify himself or herself as a member of a community 
volunteer group interested in the needs of children, and state that the 
purpose of the contact is to collect information about the services available 
for children in thearea. You need not avoid any mention of agency accountability, 
but you do riot need to'be hostifeand'abusive. A hostile approach may be 
useful in getting action but is unlikely to gain much information. 

Process of Data Collection 

Five basic types of data collection techniques have been reviewed and 
described, along with resources which must be organized to use the 
technique, the process of actually collecting the data, and advantages and 
disadvantages . The process of data collection involves the following tasks: 

1) Select appropriaietechnique{s)forcolleciing data, considering(a)iheiype 
of information desired, (h) the frajuency of monitoring desired, and (c) the 
lime of the AMG members available for monitoring. 

2) Design instruments or forms for each technique. The type of questions 
included should consider (a) the indicators selected in Step III and (b) if 
questioning of clients or agency representatives will occur. 

3) Choose an appropriate sampling procedure{see Appendix B) if client records 
or contact will be used. 

4) Collect the data according to schedule using the forms developed. 

5) Insure that accurate and reliable data are collected. 
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TABLE 6 

Recommended Data-Collection Techniques According To Type Of 
Information Desired And Type Of Monitoring Selected 



DATA-COM.ECTION 
TECHNIQUES 


MANAGEMENT 


DIRECT 
SERVICE 


EFFECT 


1. Public Documents and 
Meetings and Issues 
Research 


Recommended for 
policy and 
planning 
information 


Mainly useful for 
service descrip- 
tions 


Not 

recommended 
unless agency 
does client 
followup 


2. Mail Surveys 


Possible but not 
recommended 


Least expensive 
but better response 
from personal 
interview 


Recommended as 
least 

expensive 
means to 
contact 
clients. 


3. Telephone Interviews 


Recommended as 
alternative to 
personal 
interview 


Preferred over mail 
questionnaire. 
Alternative to 
personal interview 


Recommended. 
Preferred over 
mail 

questionnaire 


4. Personal Interviews 


Highly 

recommended but 
time consuming 


Highly 

recommended 


Highly 

recommenced but 
very time 
consuming 


5, Agency Records 


Very useful but 
difficult to 
obtain access 


Very useful for 
determining service 
flow, costs, 
waiting times, etc. 


Useful if 
agency follows- 
up on clients 
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STEP V. SCOPE AND 
OF MONITORING 



To carry out monitoring effectively the AMG must consider a great many 
things. Steps I through IV havereviewed the ingredientsof monitoring and 
many of the decisi^ms to be made. A primary concern is how to best utilize 
the time of AMG members in monitoring. 

This step, which is affected by other decisions, is to determine the 
extensiveness or scope of monitoring. Scope means how frequently 
monitoring is carried out and the amount of information obtained. For 
convenience , a suggested way to consider scope is shown in the first column 
of Table 7. Scope is categorized as light, moderate, or in-depth. Each 
category is describedby the type of data-gathering activities employed, The 
least time-consumingactivitiesare shown under *Might'' and the most time- 
, ^ consuming under **in-depth.'' Of course, the type and amount of 

information collected affect time', but the most important factor is the 
technique employed to collect information and the frequency of monitoring. 
Three possible monitoring periods (monthly, quarteriy, and annually) also 
are shown in the table. 

In the column below each time period is an estimate of total hours 
per year required for monitoring each agency selected.* As an example, 
assume we prefer **in-depth monitoring'' on a quarteriy basis. The time 
required is estimated at 50 to 80 hours per agency per year. 

Estimating the Time Required for Monitoring 

Four tasks should be completed to estimate the time required for 
monitoring: 

1. Determine the scope (light, moderate or in-depth) and frequency 
(monthly, quarteriy, or annually) of monitoring. (See the description in 
Table?.) 

2. Determine the total time required for monitoring. Use the table to 
determine the estimated time per agency. Multiply the estimated hours per 
agency times number of agencies involved. 

3. Estimate the total hours available to monitor by AMG participants. 
Multiply average hours per week per person times 50 or 52 weeks times 
the number of people involved. This will give an estimateof the total hours 
for all AMG participants. 

4. Make adjustments in the scope and/or frequency of monitoring such 
that sufficient monitoring time will be available. (Steps 3 and 4 can be 
taken in reversed order if desired.) 

An Example of Time Estimation Process 
In our example, let us say you have: (1) chosen Goals I and IV for 
monitoring, (2) identified the corresponding service areas, (3) identified the 
appropriate agencies providing such services (let us say there are a total 
individual agencies), and (4) decided to conduct in-depth monitoring 

ifMi^ C A T***** estimates arc based on the cxpcricncc.of others in collecting information about child services and are shown 
mmmmwm Q ^ here as nile-of-thumb estimates. In practice an AMG may find these estimates too high or low for a particular situation. 



TABLE 7 

Chart Of Scope Of Monitoring And Frequency, Showing 
Estimated Total Hours Annually Per Agency 



FREQUENCY OF MONITORING PER AGENCY 



Scope of monitoring per Agency 



Monthly 



Quarterly 



Annually 



I. Light monitoring: 

Only ba^ic information (covering only one 
category: management, direct service, 
or etYect) collected usually via the 
least time-consuming techniques. 
Extensive use of information which 
is genenilly available and easily 
obtainable such as through public 
records and meetings. Limited 
direct contact with agency. No 
client follow-up. 



From 48 to 
60 hours per 
agency per 
year 



From 20 to 
30 hours per 
agency per 
year 



Not 

Recommended 



IL MiHlerate monitoring: 
Collection of more infoiniation abou\ 
agency activities. (No more than 
two categories). Use readily 
available information plus 
direct contact with each agency. 
Primarily use information which 
agency staff provides. Some 
client follow-up. primarily 
using mail questionnaires and 
phone calls. 



From 60 to % From 30 to From 10 

hours per 50 hours per to 20 hours 

agency per agency per per agency 

year year per year 



II L In-depth monitoring: 

In-depth collection of information about 

agency activities including use 

of in-depth interviews with staff 

and informed others and review 

of public records of agency. 

Extensive client follow-up using 

personal interviews augmented by 

mail questionnaires and phone 

calls. 



From 96 to 
180 hours 
per agency 
per year 



From 50 to 
80 hours 
per agency 
per year 



From 20 to 
40 hours 
per agency 
per year 



NOTE:AII figures shown are the total estimated hours for one year per agency. As stated in the planning 
chapter, unless an average of five hours per month is available for all activities involving monitoring, 
then monitoring is not recommended. These estimates are averages and assume that more than 
one agency will be involved. If only one. then hours should be increased by 50 percent. 



on a quarterly basis. UsingTable?, you learned that from 50to80 hours 
annually per agency would be required to carry out this monitoring. If 10 
agencies are identified and you take 50 hours (the most conservative 
estimate), the total hours required for 12 months of monitoring would be 
500 hours. Next, you want todetermine the total hours available by AMG 
participants for monitoring. Either estimate the average time each person 
can spend each week or month. The total hours per year equals this average 
time multiplied by the number of weeks (or months) multiplied by the 
number of people: 

Total Hours Available - Average Time (Per Person x 
Number of Weeks (Or ^^nth^)^ Number of People 



Thus, if in our example you have two volunteers who estimate they can 
spend an average of 10 hours per month monitoring, then 10 hours x 2 people 
X 12 months = 240 hours. Obviously , in your planning, you have a problem 
(i,e., only 240 hours are committed to monitoring but at least 500 hours 
are required). Task 4 is to reconcile any differences by either (a) getting 
more volunteers, (b) getting more time from the existing volunteers, or (c) 
adjusting the required monitoring. In this example we will reduce the time 
required for monitoring, since it appears that with only two volunteers you 
will be able to do less monitoring than you had initially planned. Since 
annual monitoring will be too infrequent, light, quarterly monitoring will 
reduce the time required from 500 to 200 hours annually. While the highest 
time estimate for light, quarterly monitoring still exceeds the time 
available, by careful planning we can keep the actual time within the limits 
our volunteers can spend. 

Use Form AMG 03 for planning the frequency and scope of monitoring 
in your community. 
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AMC08 

WORK CHART FOR DETERMINING FREQUENCY OF MONITORING 



la/ Check category of monitoring intensity you are planning to use: 

□ Light 

□ Moderate 

□ In depth 

lb/ Check the frequency of monitoring you prefer: 

□ Monthly 

□ Quarterly 

□ Annually 

Other (specify ) 

2a/ What is the time estimate per agency forthecatcgot y of monitoring 
intensify and frequency you have selected (see Table 7)? 

From to hours per agency 

2b/ Calculate the total hours for monitoring yearly by multiplying the 
total number of agencies identified times the total number of hours 
per agency you prefef to use (the lowest estimate is the least amount 
possible and the highest estimate gives some room for unexpected 
time requirements). 

# of agencies X total hours per agency 

= total hours for monitoring annually. 

3/ Calculate the total available hours for monitoring. 

# of AMG participants X total hours per week (or month) 

X 52 (or 12) = total hours available 

for monitoring. 

4/ If 3 is greater than 2b, then go back and redo tasks 1*3 if more 
monitoring is desired. If 2b is greater than 3> then 

(a) get more people for monitoring, (b) get more time from people or 
(c) change the monitoring intensity and/or frequency. Recalculate 3. 
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STEP VL WHAT DO YOU DO 
WITH THE DATA? 

Data Summary And Analysis 

If you carry out agency monitoring (even 'light'' monitoring)aconsiderable 
amount of information will be collected. This information should be 
summarized and simplified so that it can be used and shared with others* 
Followingare some suggestions about summarizing information from each of 
the thr^e types of monitoring reviewed in Step III. 

Monitoring Service Effectiveness 

In Step III, the level ofconcern for monitoring (whether community or a 
specific agency) was designated, the desired effects (in terms of reducing 
unmet needs for children), and the indicators of effect were .selected , using 
Planning Form AMG 04. 

Onc€ these data have been collected , they Civn be organized to detect • 
problems and for use in reports to the agency(ies) and/or the public. 

Examples of ways to organize and display effectiveness data are shown 
in Table 8 for both an agency (Public Health) and a community need for all 
children no matter what agency served (medical care for children). 
In each example: 

1. The level ofconcern is identified (agency name or community need). 

2. The desired effect is restated from the planning form AMG 04. 

3. A table of desired effect in terms of numbers or percentages or averages, 
etc., is shown. 

4. The actual effect based on the data collected during monitoring is shown. 

With these elements, you have a convenient and simple way to identify 
problems. For example, the low inoculation levels fordiptheria and 
measles is shown for public health clients, and the greater percentages of 
incidence of measles among non*white children in the community or 
neighborhood is easily seen. 

In addition , you may want to show how you got the data. This allows 
others to understand where the data came from and to determine if they can 
trust your results, particulariy if you are pointing out problems. 

Monitoring Direct Services for Children 

Many numbers and figures could be collected by monitoring direct services 

for children. This detail can be * verwhelming, particularly if monitoring 

was carried out monthly. Followingare some suggestions for summarizing 

data for the characteristics of direct services introduced in Step III: 

service requirements, service-community relationships, and service 

activities. 

Service requirements include the formal guidelines, restrictions, and 
mandates placed upon agencies by the law, the fund source , or the agency 
board or management through its policy . A good way to summarize is by 
comparing what should be done with what is actually done. For example, 
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TABLE 8 

Examples Of Effectiveness Reports 

!. Agency Name: Puhlic Health Department 
11. Desired EtTcct and Results: 

''Children will have high resistance for the following diseases through 100% inocula- 
tions." 

Disease Desired Inoculations Actual Results* 

(based on monitoring) 

a. smallpox 100% 90% 

b. typhoid 100% 60% 

c. diptheria 100% 40% 

d. measles 100% 25% 

e. polio 100% 70% 
♦Based on a review of all child records for the agency. 

I. Community Need: Freedom from measles infection 

II. Deslied Effect and Results: 

•'Children between the ages of 6- 14 will haveO% incidence of measles during the year." 

Child Group Desired Incidence Actual Incidence* 

1. Black 0% 15% 

2. White 0% 4% 
y. Other 0% 10% 
*Based on interviews with a 209? sample of all school children 6-14 years old. 



one could list the requirements and check off those which are actually 
fulfilled in practice or designating whether each requirement is **always 
fulfilled/' ''sometimes fulfilled,'* or *'never fulfilled." This is shown in 
the example below for a drug counseling program: 



Program: DRUG ACTION PROGRAM 



Service Requirements 



1. Slate law requires 
that any age youth 
can obtain counseling 
services. 

2. The policy and 
procedures manual of 
the program states 
that counseling 
services will be 
available whenever 
requested by youths. 



Actual Practice 

Always 
fuiniled 



□ 



sometimes 
fulfllled 

□ 



X 



never 
fulfilled 

□ 



□ 



Alternately, a narrative report could be used to summarize requirements 
O 5 and the degree to which they are fulfilled. 



TABLE 9 

Example Of A Report Of Service^Cotnmunity Relationships 

Agency: Hilltop Center For the Retarded 
Agency: 2005 Jamestown Road 
Agency: Mitchell. Texas 

Each Year 

Maximum Number of Capacity • Actual Number 
Children Needing Service Served 



1. Programs: 

1. Long-term residental care 200 100 120* 

2. Non-residental day 

care 600 200 160** 

3. Short-term (90 day) 

residental care 90 20 30* 



II. Deterrents to Service: 

1. Cost is high for low-income families so that disproportionate numbers of higher-income 
families are represented 

2. Need daily transportation for day-care program 

'allowing for turn-over in residents < 
••program is not well known to community 



Service-Community relationships concerns the openness and relevancy 
of services to the needs of children. The monitoring forms AMG 05, used 
to collect data, and AMG 06, used as an analysis form, were shown in 
Step III. The basic intent of monitoring service-community relationships 
is to determine how the actual services match with what is needed, i.e., 
can the program serve all children who need the service and are there 
deterrents which prevent children from obtaining service? An example of 
such information in table form based on monitoring a state-supported 
residential program for mentally retarded children is shown in Table 9. 
Such a summary shows those programs operating at capacity and one (the 
day-care program) operating at under-capacity with a significant- 
community need. The advocate has also identified the large unmet need in 
the community for more services for retarded children, particularly non- 
residential services. 

Service activities are. those which relate directly to children. Form 
AMG 09 provides guidance in summarizing data about services gathered 
in monitoring. There are sections of the form which relate to: 



a) Client Volume and Flow 

b) Service Responsiveness 

c) Types of Service Activities 

d) Service Resources 

e) Service Costs .-^ 



Monitoring Agency Management 

Important considerations in reviewing and evaluating the management 
activities of services for children are: 

1) Performance - whether the aaivity is actually carried out and if it benefits 
children via high quality services. 

2) Sensitivity - are management activities open to and informed about the 
needs, experiences, and ideas of children and their advocates? 

3) Problems and/or Recommendations - does monitoring identify any problems, 
and can specific recommendations be made? 

An example of how these considerations could be summarized is shown in 
Table 10. This example gives the high points of monitoring at a community 
day care center. Activities which exist and how well they are carried out 
are under "performance.'' Whether the activity involves clients or is open 
to parents is documented under ^^sensitivity." Recommendations which the 
AMG has for management are shown in the last column. In this example, 
the center lacks service planning and a written version of policies and 
procedures (thus parents and staff are often 'left in the dark" about what 
to do). The staff does include parents in many of its decisions and is open 
to change. No follow-up of clients is carried out. 

An alternate way to summarize information about management is in a 
report using a narrative style. The report could describe the various 
activities of management, who the decision-makers were, how monitoring 
was carried out (as in the above illustrations), assess performance and 
sensitivity, and make recommendations. 

Summary 

Step VI has outlined ways in which the data collected can be analyzed 
and summarized. The volume of data and the plans for use of the data will 
determine the form to be taken. Professional assistance can be used during 
this step to organize the particular data in the most meaningful way. 
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SUMMARY FORM FOR SERVICE ACTIVITY MONITORING 



Client Volume and Flow 

Monthly Quarterly 

# of Child Cases: Jan Feb , , , 12 3 4 



1/ Accepted for 
Service 
















2/ Closed due 
to completion 
















3/ Closed due 
to referral 
















4/ Who dropped 
out 
















5/ Requesting 
service but 
not accepted 
















6/ Closed and 
followed up 

















Service Responsiveness 

1/ Lengthoftimefromenrollmenttoservicecompletion = , 

21 Length of time from initial request to beginning of actual service 
delivery = 

3/ Frequency of service delivery to children: 

(a) (b) 

Daily □ Desired Frequency Actual Frequency 



Weekly □ 
Monthly □ 
Other □ 



Maximum number of 
children who can 
be served in one 
day == 




Average number of 
children actually 
served in one day = 
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5/ Are the rights of children monitored by the staff?: 
yes no don't know — 



Types of Service Activities 

Name the services or programs which fall under each of the four service 
components: 

a) Intake and assessment 

b) Individual Client 
Service Planning 

c) Direct services 

d) Individual follow-up 



Service Resources 

1/ Staff Positions # of Staff Relevant Experience 

and Training 



1. 






2. 






3. 






4. 






5. 







# of children served 
21 Client/Staff Ratio = # ofdirect service staff = — 
3/ Total annual budget for service or program = $ 



Service Costs 

1/ The major unit of service is: . 

(For example, residental or hospital day of care, or outpatient 
hour or day of care, or per child care, or average daily attendance, 
etc.) 

21 The total units of service delivered per year = 

3/ The cost per unit of service is $ . 



Q (Calculated by dividing the total annual budget for service or 

FR \ program by the total units of service delivered (#2)). 



TABLE 10 

Example Of Summarizing Management Activities Based On Monitoring 

Agency: Child Day Care Center 
Agency: 211 S. Main Street 
Agency: South Jasper, Ohio 

Management Performance Sensitivity Problems and/or 

Activities Recommendations 



1. Planning 



2.Decisions about 
policy and 
procedures 



No formal service 
planning exists. 



No written 
statement of 
policy or 
procedures for 
staff or clients. 



Agency listens 
to problems & 
concerns of 
parents in 
selecting new 
programs. 

Parents are not 
involved in 
setting policy. 



3. Resource 
acquisition 
and allocation 



Budget available for 
review. 

Staff is experienced 
with day care activities 
but lacks training with 
behavioral problems or 
handicapped children. 



Parents are 
asked to review 
staff performance. 



Recommend 

planning 

at least annually 



Recommend written 
manual of policy 
and procedures 



Recommend at least 
semi-annual survey 
of clients 



4. Evaluation and ^ 
modifications 



Staff does not 
routinely follow-up 
children served. 
Problems are 
corrected when 
found for individual 
children. 



Parent concerns 
& complaints are 
important to staff. 
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WHAT ACTION CAN BE TAKEN? 

Possible Uses Of Information Obtained 
In Monitoring 



The main purpose of this handbook is to assist the process of agency 
monitoring, i.e., the collection of information. However, some possible 
results of using the information collected are summarized below. Other 
uses exist, and each community will find applications not contained here. 

Often advocacy groups are handicapped because they do not have 
facts or data to support the actions they advocate. The monitoring process 
described here can produce specific information which can be the basis of 
action for child advocates and can produce results described below. 

Increased Agency Accountability 

The use which first suggests itself is to hold agencies accountable for 
problems or inadequacies in service, failures to meet minimum standards 
or respect the rights and needs of children, or detrimental effects for the 
children served. 

If agency maragement activities are inadequate, or ineffective, 
intervention should be made at the agency governing level by informing the 
agency direcicr, governing boards, and/or local, regional, state, or federal 
regulatory organizations. In reporting about agency management, describe 
the situation and document how the situation was identified, when and by 
whom (show as much specific data as possible). Point out why this is a 
problem and what the consequences have been and/or will be in the future. 

If the problem is service delivery, contact agency management first 
to seek, cor recti on and improvement. If this does not work, contact the 
groups suggested for reporting management problems. Remember that many 
programs of service are accountable for meeting certain minimum standards 
established by regulatory agencies and/or funding sources. If these 
regulations are violated, agency management and the regulatory agency 
should be contacted jointly and a correction requested. Court action could 
be necessary and would be appropriate if violations are not corrected 
when identified. 

When services meet minimum standards but are being delivered 
incorrectly or inappropriately, with unnecessarily high costs, in 
undesirable conditions, and/or by insufficiently trained or inappropriate 
personnel, agency management should be informed. Unlike the violation of 
regulations, these types of problems are not as easily identified and are 
subject to interpretation. Consequently, careful documentation of all 
pertinent information is essential. 

If clients are dissatisfied or if the effectiveness of services is not 
as desired, a report showing the results obtained by monitoring clients 
sent to the agency, the news media, and state regulatory agencies would be 
appropriate. Such information can be used by consumers to obtain 
corrective action or redress for ineffective service. It can stimulate an 
gl^^-l agency to better planning: . ^ ^ j) § j 



In general, no matter the type of information, the following steps are 
appropriate for insuring accountability: 

1) identify the person(s) or organization or groups most likely to act on the 
information, 

2) describe the situation or findings, 

3) document the ways this information was obtained, when and by whom, 

4) describe how the situation is a problem and its past and future consequences, 
and 

5) suggest ways to improve or correct the situation. 

A Published Child Service Guide 

One important product of monitoring could be the development and 
distribution of a guidebook of community services for children. Such a 
guide could be useful to consumers in seeking service and to agency 
professionals in making referrals. It could improve agency collaboration 
and coordination. The initial agency inventory completed in Step 1 would 
be useful in itself, but an expanded version could be developed using 
information obtained from monitoring. The expanded version could give 
consumers more information for selecting services. 

Documentation of the Network of Child Services 
Going beyond the development of an inventory, one could use the 
information obtained through monitoring to document and describe in detail 
the overall network of community services for children. 

Such documentation may be extremely useful to a child advocacy group 
in identifying points in the network which requires attention or careful 
scrutiny. Further, agencies themselves can use the information for 
planning, identifying service gaps, improving the delivery of services 
through a better use of community resources, and initiating their own 
evaluation processes. 

Identification of Unfilled Needs and Service Gaps 
While monitoring is directed at insuring accountability for existing 
agencies, the information can be used to identify child needs to which no 
service is addressed, i.e., where service gaps exist. By organizing data 
about all available services into a summary form, gaps v/ill often readily 
appear which are not obvious when individual agencies are considered. A 
summary of services for children should indicate types, quantity, 
availability, and location. The summary is an aggregation and emphasizes 
overall availability and not individual services. 

Development of Better Records and Self-Accountability 
Much of the information obtained in monitoring is not maintained by most 
service agencies. Consequently, the processes and techniques suggested in 
this guide could be used by agencies as the basis for developing an improved 
record system for themselves. Few service agencies have readily available 
the information necessary to monitor for themselves the quality and 
quantity of their services, -> ^ ^ a a 




It is not unreasonable to believe that regular monitoring by child 
advocacy groups will stimulate agencies previously unconcerned with 
accountability to become involved in self-monitoring. Increasing the 
involvement of community residents in services for children through 
monitoring would make a significant contribution to the quality of 
community quality life. Both citizens and agencies staff will benefit 
from stronger citizen participation. 
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AMGoiA APjPENDIX A 

Child Service Agency Description 



(Auxilliary Form to be used with Form AMG 01 for each agency included 
in the Child Service Agency Inventory) 

I. Using the list of services indicated for an agency on form AMG 01, 
give a brief description and name of the program(s) providing each 
service. 

II. Describe the child population(s) the agency serves and any limitations 
or restrictions by physical, mental, economic or other characteristics 
for each service and/or program in the agency. 

III. Describe and document any fees for the service(s). 

IV. The total agency budget is $ 

Identify the budget for each of the child services and/or programs. 
Show the sources for funds for the agency and/or each service or 
program and the percentage each source contributes: 



Federal Government % 

State Government % 

Municipal/County Gov't. % 

Private Contributions, 

or gifts, including 

Community Chest or 

United Way % 

Client Fees % 

Other % 



Total 100 % 
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APPENDIX B 

Sampling Procedures Or How To Get Accurate 
Information Without Contacting Every Client 

When one wishes to get information about a large group of people, there 
are two alternatives, (1) contact every one (or review the available 
records of each) or (2) sample a small representative group. Sampling is 
used in public opinion polling, advertising and market research, and TV 
ratings. A carefully selected set of individuals are interviewed or polled 
and their answers are generalized for an entire nation, region, state, or 
community. Through this means, reliable information is obtained quickly 
at a relatively low cost. The same benefits also apply to obtaining 
information about clients of services. 

Unless the number of clients served is less than 50, sampling is 
strongly recommended. The technical and mathematical background of 
sampling is not needed. Some straightforward procedures for sampling 
will be outlined here. 

By definition a sample group should be representative of the whole 
group. All techniques for^sampling must be evaluated according to their 
ability to obtain information representative of all clients receiving a service 
Following are some sampling techniques which, if applied properly, can 
produce reliable information. 

Random sampling - In this technique all clients have ''equal 
opportunity** of being selected for contact or record review. This is 
similar to pulling numbers out of a hat as in drawing for a prize. First, 
a list of ali clients is compiled and each client is assigned a sequential 
number. Second, to insure equal opportunity of selection, a table of random 
numbers is used to select those clients from the list who will compose the 
sample. Tables of random numbers are found in statistics books and 
library reference shelves. The table of random numbers is scambled 
already so that every number has an equal chance of selection no matter 
where you start in the table. The clients selected for the sample will 
therefore be representative of ^he whole group. If the clients are not 
listed but occupy an entire area such as a neighborhood, houses can be 
used as sampling units and numbered. The selection process is the same. 
The chief advantage of a random sample is the likelihood that representative 
information of the whole group will be obtained. Disadvantages include 
the difficulty of obtaining an entire client listing and thus the cost and 
time required in selecting such a sample. 

Stratified Random Sampling - This sampling technique, a version of the 
simpie random selection above, is employed when one suspects that certain 
subgroups (or strata) of a client population may be overlooked. Therefore, 
the entire client group is divided into subgroups (for example, into 
physically handicapped and non-handicapped or poverty and above poverty 
family groups) and random selections are made from each subgroup 
„ using the random number table. It is important that the size of the sample 
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drawn from each subgroup be proportional to the subgroup's size in the whole client 
population. That is if physically handicapped children are 12% of the entire client 
population , then their sample should be 12 % of the entire sample. This technique 
has the same advantages as random sampling plus the added advantage of 
insuring that important child subgroups do not get overlooked. Same 
disadvantages apply. 

Systematic Sampling - This technique uses a list of clients or a set of 
client records or even the flow of clients Ihrough a direct service. It 
draws every ''nth" client, c.^^'., every ''5th'' or "10th" or "25th/' 
This procedure does not require a complete listing and numbering of clients. 
The selection is simple (not requiring the thoroughness of random 
sampling). This technique is reliable (i.e», the sample selected will be 
representative of the whole group) if there is no bias in the ordering or 
sequencing in which clients are listed, their records stored, or they appear 
for service. Just as with random sampling, every client should have equal 
opportunity of selection. To the degree this is true, a representative 
sample can be selected. If, for example, only the poor come for service 
on Mondays and Friday and you are interviewing clients on other days, or 
the records of certain clients are not kept in the files, then the sample will 
be biased. Chief advantage is the simple procedure for selecting the sample. 
Main disadvantage is its vulnerability to bias in the way that client names 
or records are available or that clients show up for service. 

Opportunistic Sampling - Opportunistic sample, as its name implies, means 
contacting clients or reviewing records which are immediately available. 
This can occur by obtaining information about clients actually getting 
service on a particular day in a waiting room or emergency room at a 
hospital, or those contacted by a case worker, etc. The chief advantage 
of this technique is convenience and ease in sampling. Chief disadvantage 
is the unrepresentativeness of such a sample. If several such samples 
are drawn with this procedure and results obtained do not substantially 
vary across samples, the procedure can be more trusted. 

Other Considerations - Other considerations in sampling include (a) the 
size of sample and (b) replacement of people who do not respond. The 
size of the sample is dependent upon the amount of time you have to monitor 
(the bigger the sample, the more time required) and the desired 
representativeness. All these things equal, the larger the sample, the more 
representative the sample. Generally, in agency monitoring a 10 to 20% 
sample would be a good rule-of-thumb. Of course 10% of 10,000 clients 
may be too big to survey, and a 1% sample better handled. 

Some rule-of-thumb suggestions are given below concerning the size of 
the sample (expressed as a percentage of the whole), depending upon the 
size of the whole group of clients: 

Sample size Client Group 

50% 50-100 
30% 100-400 

Q 10% 400 and above 
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Each person in a sample represents many others in the whole group. 
For each sample person who does not answer or is not contacted, all those 
he represents are overlooked. Therefore, it is important that a replacement 
is selected for each uncontacted sample member. The %ame sampling 
technique used to draw the original sample can be used to select 
replacements. 
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